Virtual reality systems are having an ever increasing impact on our daily lives. Virtual world games, for example, continue to dominate the gaming industry and are migrating to mobile devices of all sorts including cell phones. Just as virtual worlds are impacting our real lives, the real world will have an ever growing impact on our virtual lives. The distinction between virtual worlds and the real world will continue to become ever more blurry resulting in an ever present mixed reality. People living with the various mixed realities will interact with mixed reality objects whether they are real, virtual, or a mix of both. To date there are no easy, intuitive methods for a person to command or control mixed reality objects.
Google Maps™ provides a method for people to interact with a mixed reality where real-world satellite images are superimposed over computer generated images showing location or road names. Although Google Maps is useful for interacting with maps and for virtual travel, it does not address command or control of mixed reality objects.
U.S. Pat. No. 6,502,000 to Arnold et al. offers a system for controlling real-world devices through a communication link between a controlled device and a controlling device but does not address the need for interacting with virtual objects of a mixed reality. The Arnold approach is useful for closed ecosystems where all devices use a common control protocol. However, the approach does not lend itself to a universal, heterogeneous environment where mixed reality objects employ multiple command or control protocols.
The mixed reality system described by U.S. Pat. No. 6,604,049 to Yokota teaches that a person can interact with a mixed reality by accessing and displaying information about a real-world object on a virtual reality display device worn by the person. A camera captures images of surrounding objects and sends the image data to a remote server for processing. The server then returns information associated with the objects for display on the user's display screen. Although useful for obtaining information about real-world objects, the system does not teach how to command or control mixed reality objects. Additionally, it is unlikely individuals will purchase or wear the contemplated virtual reality system as opposed to using a system the individual already owns, possibly a cell phone or PDA.
U.S. Publication No. 2002/0090985 to Tochner et al. further advances mixed reality interactions by teaching that a player of a virtual world game can interact with the real-world by establishing a correspondence between virtual objects and real-world objects. Still, Tochner does not provide for the command or control of mixed reality objects.
The applicant's U.S. Pat. No. 7,016,532 and patent applications U.S. Publication Nos. 2004/0208372, 2006/0002607, 2006/0110034, and 2006/0181605 teach that one can use a mobile device to capture image data associated with a real-world object to access information about the real-world object. The image data is used to identify network addresses where information associated with the real-world objects can be obtained. The network addresses can also be used to perform an action relating to the real-world objects. Although the applicant's previous filings teach performing an action associated with a returned network address and controlling real-world objects, those filings do not teach how to command or control mixed reality objects.
What has not yet been appreciated is that a mobile device can provide an interface to command or control mixed reality objects by capturing an image of a real-world object. Thus, there is still a need for systems and methods for interacting with mixed realities.